Papers with retrieval models
Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face (2023.emnlp-demo)
Copied to clipboard
Christopher Akiki, Odunayo Ogundepo, Aleksandra Piktus, Xinyu Zhang, Akintunde Oladipo, Jimmy Lin, Martin Potthast
| Challenge: | a toolkit for reproducible information retrieval research is available for free. |
| Approach: | They present a tool that integrates Pyserini and Hugging Face to enable the seamless construction and deployment of interactive search engines. |
| Outcome: | The proposed tool makes state-of-the-art retrieval models more accessible to non-IR practitioners while minimizing deployment effort. |
Improving Embedding-based Large-scale Retrieval via Label Enhancement (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for large-scale retrieval are trained with 0-1 hard labels that indicate whether a query is relevant to a document, ignoring rich information of the relevance degree. |
| Approach: | They propose to introduce label enhancement for the first time to characterize query-document relevance degree by embedding label distribution into contextual embeddables. |
| Outcome: | The proposed method significantly outperforms existing retrieval models and its counterparts equipped with two alternative methods on English and Chinese large-scale retrieval tasks. |
Out of Style: RAG’s Fragility to Linguistic Variation (2026.eacl-long)
Copied to clipboard
| Challenge: | linguistic reformulations impact both retrieval and generation stages, leading to a relative performance drop of up to 40.41% for less formal queries and 38.86% for queries containing grammatical errors. |
| Approach: | They evaluate two retrieval models and nine LLMs across four QA datasets and examine how linguistic reformulations impact RAG performance. |
| Outcome: | The proposed models show that linguistic reformulations significantly impact both retrieval and generation stages, leading to a performance drop of up to 40.41% for less formal queries and 38.86% for queries containing grammatical errors. |
Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Early fusion models with cross-attention have shown better-than-human performance on some question answer benchmarks, while it is a poor fit for retrieval since it prevents pre-computation of the answer representations. |
| Approach: | They propose a supervised data mining method to train an efficient late fusion retrieval model by using cross-attention models with cross-references. |
| Outcome: | The proposed model outperforms retrieval models trained with gold annotations on Precision at N (P@N) and Mean Reciprocal Rank (MRR). |
GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to perform large-scale query-passage retrieval are term-based, but they lose interaction between query-pastage pairs. |
| Approach: | They propose to fuse query (passage) information into query representations via graph neural networks that are constructed by queries and their top retrieved passages. |
| Outcome: | The proposed model outperforms existing models on MSMARCO, Natural Questions and TriviaQA datasets and achieves the new state-of-the-art on these datasets. |
Target-Guided Open-Domain Conversation Planning (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on goal-oriented conversational tasks lack planning . prior studies on this topic have focused on generating proactive behavior in agents . |
| Approach: | They propose a task to evaluate whether neural conversational agents have goal-oriented conversation planning abilities. |
| Outcome: | The proposed task evaluates whether neural conversational agents have goal-oriented conversation planning abilities. |
Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Adapting recipes to cultural differences presents significant importance and challenges . bridging cultural differences is a challenge, but IR can help. |
| Approach: | They propose a framework that preserves the original recipe and its cultural appropriateness for the target culture. |
| Outcome: | The proposed framework preserves the original recipe and its cultural appropriateness for the target culture while maintaining relevance to the original. |
MindRef: Mimicking Human Memory for Hierarchical Reference Retrieval with Fine-Grained Location Awareness (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods require pre-segmented article chunks, limiting reference flexibility like human memory. |
| Approach: | They propose a framework that leverages parameterized knowledge stored during the pre-training phase of large language models to recall reference passages from any starting position independently. |
| Outcome: | The proposed framework can recall reference passages from any starting position independently. |
Efficient and Versatile Model for Multilingual Information Retrieval of Islamic Text: Development and Deployment in Real-World Scenarios (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Despite recent advances in multilingual information retrieval, a significant gap remains between research efforts and real-world deployment. |
| Approach: | They propose to use Quranic multilingual corpus to develop an ad-hoc IR system that can satisfy users’ information needs in multiple languages. |
| Outcome: | The proposed model achieves promising results across diverse retrieval scenarios. |
DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management (2025.findings-emnlp)
Copied to clipboard
Kai Yin, Xiangjue Dong, Chengkai Liu, Lipai Huang, Yiming Xiao, Zhewei Liu, Ali Mostafavi, James Caverlee
| Challenge: | Existing information retrieval benchmarks focus on general or specialized domains, such as medicine or finance, neglecting the unique linguistic complexity and diverse information needs encountered in disaster management scenarios. |
| Approach: | DisastIR is the first comprehensive IR evaluation benchmark specifically tailored for disaster management. |
| Outcome: | DisastIR covers 48 retrieval tasks derived from six search intents and eight general disaster categories . evaluations show no single model excelling universally . |
Efficient Passage Retrieval with Hashing for Open-domain Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Open-domain question answering systems often require large memory to run because of the massive size of their passage index. |
| Approach: | They propose a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever to represent the passage index using compact binary codes. |
| Outcome: | The proposed model significantly reduces memory cost from 65GB to 2GB without loss of accuracy on two open-domain question answering benchmarks. |
D.Va: Validate Your Demonstration First Before You Use It (2025.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) heavily relies on selecting effective demonstrations to achieve outputs that better align with the expected results. |
| Approach: | They propose a method which integrates a demonstration validation perspective into this field and integrates it into the learning paradigm. |
| Outcome: | The proposed method surpasses all retrieval-based in-context learning techniques across both natural language understanding (NLU) and natural language generation (NLG) tasks. |
ColMate: Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval (2025.emnlp-industry)
Copied to clipboard
Ahmed Masry, Megh Thakkar, Patrice Bechard, Sathwik Tejaswi Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte Suresh, Srivatsava Daruru, Enamul Hoque, Spandana Gella, Torsten Scholak, Sai Rajeswar
| Challenge: | Existing methods for multimodal document retrieval often replicate techniques developed for text-only retrieval. |
| Approach: | They propose a document retrieval model that bridges the gap between multimodal representation learning and document retrievals by providing external knowledge as context. |
| Outcome: | The proposed model achieves 3.61% improvement over existing retrieval models on the ViDoRe V2 benchmark, showing stronger generalization to out-of-domain benchmarks. |
ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links (2026.eacl-long)
Copied to clipboard
| Challenge: | Using retrieval models and LLMs achieves a 73% approval rate for suggested links, more than doubling the acceptance of strong retrievers alone. |
| Approach: | They propose a domain-agnostic framework for bootstrapping sentence-level cross-document links from scratch and apply it to large-scale human-in-the-loop annotation of natural text pairs. |
| Outcome: | The proposed framework generates semi-synthetic datasets and uses them to benchmark and shortlist the best-performing methods and applies them in large-scale human-in-the-loop annotation of natural text pairs. |
PAMN: Multi-phase Correlation Modeling for Contrast-Enhanced 3D Medical Image Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current 3D medical imaging models focus on spatial features, neglecting phase-specific progression detailed in clinical reports. |
| Approach: | They propose a framework that fuses imaging phases with clinical text to enhance 3D medical image retrieval. |
| Outcome: | The proposed framework outperforms state-of-the-art models on a phase-series dataset of 12,230 hospital CT scans. |
Benchmarking and Building Zero-Shot Hindi Retrieval Model with Hindi-BEIR and NLLB-E5 (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for evaluating retrieval models in Hindi are lacking . despite efforts to build multilingual retrieval systems, this is still a work in progress . |
| Approach: | They evaluate Hindi retrieval models on the Hindi-BEIR benchmark and introduce a multilingual model that leverages a zero-shot approach to support Hindi without the need for Hindi training data. |
| Outcome: | The proposed model leverages a zero-shot approach to support Hindi without the need for Hindi training data. |
JuriFindIT: an Italian legal retrieval dataset (2026.findings-eacl)
Copied to clipboard
| Challenge: | Statutory article retrieval (SAR) targets retrieval of legislative provisions relevant to a natural language question. |
| Approach: | They propose a pipeline that integrates dense encoders with an heterogeneous legislative graph . they propose statutory article retrieval (SAR) is the first SAR dataset for the italian legal domain . |
| Outcome: | The proposed pipeline improves over existing approaches. |
Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented diffusion models (RDMs) have been developed to enhance performance with reduced parameters. |
| Approach: | They propose to integrate retrieval-augmented diffusion models with Retrieval-augmented generation (RAG) that enhances performance with reduced parameters. |
| Outcome: | The proposed framework achieves outstanding attack effects while maintaining benign utility. |
RetrieverGuard: Empowering Information Retrieval to Combat LLM-Generated Misinformation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in generating human-like text and can store factual knowledge within their extensive parameters. |
| Approach: | They propose a self-supervised training method that captures textual rules and styles of false information from the corpus without human-labelled data. |
| Outcome: | The proposed method can capture rules and styles of false information from the corpus without human-labelled data, achieving higher accuracy and robustness in identifying misleading and highly deceptive AI-generated content. |
Beyond the Surface: A Solution-Aware Retrieval Model for Competition-level Code Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrieval models emphasize surface-level semantic similarity, neglecting deeper solution-level logical similarities. |
| Approach: | They propose a solution-aware ranking model empowered by synthetic data for competitive programming tasks. |
| Outcome: | The proposed ranking model outperforms existing retrieval models in precision and recall metrics. |
Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | generative models are less practical for building real-time conversation systems due to high latency and large memory footprint. |
| Approach: | They propose a method that preserves the efficiency of a retrieval model while leveraging the conversational ability of generative models. |
| Outcome: | The proposed method preserves the efficiency of a retrieval model while leveraging the conversational ability of generative models. |
Know When to Fuse: Investigating Non-English Hybrid Retrieval in the Legal Domain (2025.coling-main)
Copied to clipboard
| Challenge: | Existing research focuses on a limited set of retrieval methods, evaluated in pairs on domain-general datasets exclusively in English. |
| Approach: | They evaluate the efficacy of hybrid search across a variety of retrieval models in the french language . they find that fusion of different domain-general models consistently enhances performance . |
| Outcome: | The proposed model improves in-domain performance compared to a single model in a zero-shot context . the proposed model also improves when the models are trained in- domain . |
Tackling Distractor Documents in Multi-Hop QA with Reinforcement and Curriculum Learning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on retrieval-augmented generation systems has shown that retrievers exhibit imperfect recall and precision, limiting downstream performance. |
| Approach: | They propose a retrieval-augmented generation model that generates answers from larger sets of retrieved contexts. |
| Outcome: | The proposed model generates answers and cites relevant information from larger sets of retrieved contexts. |
EDIS: Entity-Driven Image Search over Multimodal Web Content (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing image retrieval methods require large datasets and a large candidate set. |
| Approach: | They propose a news-domain dataset for cross-modal image search with 1 million web images . they propose combining multimodal image-text pairs with a million candidates . |
| Outcome: | The proposed dataset challenges state-of-the-art methods with dense entities and the large-scale candidate set. |
Decomposing Complex Queries for Tip-of-the-tongue Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Tip-of-the-tongue retrieval is a retrieval setting in which a user is unable to formulate a precise query that identifies a sought item . a framework that decomposes complex queries into subqueries can improve gold book recall . |
| Approach: | They propose a framework for handling tip-of-the-tongue queries by decomposing queries into individual clues routing them to specialized retrievers. |
| Outcome: | The proposed framework improves gold book recall up to 6% on a new query-book pair . it takes advantage of off-the-shelf retrievers or incorporates retriever-specific logic . |
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new benchmark for natural language inference is proposed for long premises . lawngNLI can train and test systems for implication-based case retrieval and argumentation. |
| Approach: | They propose a new natural language inference benchmark LawngNLI from U.S. legal opinions with automatic labels with high human-validated accuracy. |
| Outcome: | The proposed benchmark can train and test systems for implication-based case retrieval and argumentation. |
Learning to Abstract for Memory-augmented Conversational Response Generation (P19-1)
Copied to clipboard
| Challenge: | Existing generative models for open-domain chit-chat conversations lack informativeness and diversity. |
| Approach: | They propose a retrieval-augmented generative model that learns to abstract from the training corpus and saves useful information to the memory to assist the response generation. |
| Outcome: | The proposed model outperforms other baselines in query-response clustering and learning to utilize these characteristics for response generation. |
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval models based on dense representations show better performance than sparse representations. |
| Approach: | They propose a method to mimic the queries to each of the documents by an iterative clustering process and represent the documents using multiple pseudo queries. |
| Outcome: | The proposed model achieves state-of-the-art results on a large dataset while remaining high efficiency. |
LaMP: When Large Language Models Meet Personalization (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for personalization in large language models are understudied . |
| Approach: | They propose a benchmark for training and evaluating language models for producing personalized outputs using a set of seven personalized tasks . they propose two retrieval augmentation approaches that retrieve personal items from each user profile for personalizing language model outputs. |
| Outcome: | The proposed approach is effective for a set of zero-shot and fine-tuned language models and highlights the impact of personalization in various natural language tasks. |
MRF-Chat: Improving Dialogue with Markov Random Fields (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deep learning for open-domain dialogue include training end-to-end models to learn various conversational features like emotional content of response, symbolic transitions of dialogue contexts and persona of the agent and the user, among others. |
| Approach: | They propose a probabilistic approach using Markov Random Fields to augment existing deep-learning methods for improved next utterance prediction. |
| Outcome: | The proposed approach significantly improves the performance of existing state-of-the-art retrieval models for open-domain conversational agents. |
Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration (2024.findings-acl)
Copied to clipboard
Sunhao Dai, Weihao Liu, Yuqi Zhou, Liang Pang, Rongju Ruan, Gang Wang, Zhenhua Dong, Jun Xu, Ji-Rong Wen
| Challenge: | Large Language Models (LLMs) have led to an influx of AI-generated content on the internet, transforming corpus of Information Retrieval (IR) systems from human-written to a coexistence with LLM-generated contents. |
| Approach: | They propose a benchmark named Cocktail that compares IR models with LLMs to find relevant documents and passages from a corpus. |
| Outcome: | The proposed benchmark aims to evaluate IR models in the mixed-sourced data landscape of the LLM era. |
Grounding Fallacies Misrepresenting Scientific Publications in Evidence (2025.naacl-long)
Copied to clipboard
| Challenge: | Current fact-checking models struggle to use misrepresented scientific passages to refute misinformation. |
| Approach: | They propose an extension of the fallacy detection dataset Missci to detect and highlight such fallacies by grounding applied fallacias in real-world passages from misrepresented studies. |
| Outcome: | The proposed model can detect and highlight misrepresented publications and verbalize them to refute them. |
Uni-Retrieval: A Multi-Style Retrieval Framework for STEM’s Education (2025.acl-long)
Copied to clipboard
| Challenge: | Current retrieval models focus on natural text-image retrieval, which is insufficient for STEM education contexts due to ambiguities in the retrieval process. |
| Approach: | They propose a diverse expression retrieval task tailored to educational scenarios . they extract query style features as prototypes and build a continuously updated Prompt Bank . |
| Outcome: | The proposed model outperforms existing retrieval models in most retrieval tasks. |
Understanding Retrieval Robustness for Retrieval-augmented Image Captioning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent retrieval-augmented models for image captioning are not perfect in practice. |
| Approach: | They propose to train a retrieval-augmented captioning model SmallCap by sampling retrieved captions from more diverse sets. |
| Outcome: | The proposed model is sensitive to tokens that appear in the majority of retrieved captions . the proposed model improves both in-domain and cross-domain performance . |
Synergistic Interplay between Search and Large Language Models for Information Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Information retrieval (IR) is an indispensable technique for locating relevant resources from vast amounts of data. |
| Approach: | They propose a framework that facilitates information refinement through synergy between RMs and LLMs. |
| Outcome: | The proposed framework improves the performance of large-scale retrieval benchmarks on web searches and low-resource retrieval tasks. |
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions (2025.naacl-long)
Copied to clipboard
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini
| Challenge: | Modern language models (LMs) are capable of following long and complex instructions that enable a large and diverse set of user requests. |
| Approach: | They propose a dataset that contains an instruction evaluation benchmark and a training set to help IR models learn to follow instructions. |
| Outcome: | The proposed model improves after fine-tuning on a training set and rigorous instruction evaluation benchmark. |
Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages (2025.coling-main)
Copied to clipboard
| Challenge: | lexical gaps between dialects in cross-lingual information retrieval (CLIR) are caused by orthographic variations and different regional expressions. |
| Approach: | They propose a dataset that consists of seven German dialects extracted from Wikipedia. |
| Outcome: | The proposed dataset consists of seven German dialects extracted from Wikipedia. |
LexCLiPR: Cross-Lingual Paragraph Retrieval from Legal Judgments (2025.acl-long)
Copied to clipboard
| Challenge: | Existing work on IR focus on retrieving entire cases rather than precise, paragraph-level information. |
| Approach: | They propose a cross-lingual dataset for paragraph-level retrieval from ECtHR judgments . they evaluate retrieval models in a zero-shot setting and use multilingual case law guides . |
| Outcome: | The proposed model excels in cross-lingual retrieval, while siamese architectures are better suited for monolingual tasks. |
M3Retrieve: Benchmarking Multimodal Retrieval for Medicine (2025.emnlp-main)
Copied to clipboard
| Challenge: | Strong retrieval models are increasingly important in knowledge-intensive domains. |
| Approach: | They propose a benchmark to evaluate multimodal retrieval models in medical settings . they examine 1.2 million text documents and 164K multimodal queries . |
| Outcome: | The proposed model spans 5 domains,16 medical fields, and 4 distinct tasks with over 1.2 Million text documents and 164K multimodal queries. |
A Comprehensive Taxonomy of Negation for NLP and Neural Retrievers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new taxonomy of negation is proposed to improve neural information retrieval models . negation types are covered in existing datasets, allowing for faster convergence . |
| Approach: | They propose a taxonomy of negation that derives from philosophical, linguistic, and logical definitions . they also propose analyzing the performance of retrieval models on existing datasets using a logic-based classification mechanism. |
| Outcome: | The proposed taxonomy produces a balanced data distribution over negation types . it also provides a better training setup that leads to faster convergence on the NevIR dataset . |
LitSearch: A Retrieval Benchmark for Scientific Literature Search (2024.emnlp-main)
Copied to clipboard
| Challenge: | Literature search questions pose significant challenges for modern retrieval systems . a lack of domain expertise and reasoning through lengthy papers is a challenge . |
| Approach: | They propose a retrieval benchmark for literature search queries using inline citations from papers and questions about recently published papers. |
| Outcome: | The proposed retrieval benchmarks outperform state-of-the-art retrieval models and reranking pipelines. |
Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt tuning is a technique that updates few parameters in pre-trained models for language understanding and generation tasks. |
| Approach: | They propose to leverage prompt tuning for neural text retrieval to improve generalization and cross-domain generalization. |
| Outcome: | The proposed approach can mitigate the two issues faced by fine-tuning retrieval methods and improve the out-of-domain zero-shot generalization of the retrieval models. |
Evaluating the Effectiveness and Scalability of LLM-Based Data Augmentation for Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing research does not explore key factors such as optimal augmentation scale and the necessity of using large augmentation models. |
| Approach: | They propose to use LLMs to augment compact dual-encoder models to improve retrieval performance. |
| Outcome: | The proposed approach improves retrieval performance but its benefits diminish beyond a certain scale even with diverse augmentation strategies. |
Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal retrieval models are lacking in visual representations of multimodal data. |
| Approach: | They propose a visualized information retrieval paradigm where multimodal information is represented by a unified visual format called Screenshots for various retrieval applications. |
| Outcome: | The proposed model is based on a large dataset of screenshots from diverse sources . it is compared with existing models and lays a solid foundation for the new model . |
RaDeR: Reasoning-aware Dense Retrieval Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | RaDeR retrievers outperform strong baselines in reasoning tasks . large language models (LLMs) have impressive reasoning capabilities on a wide range of tasks - however, they face challenges when reasoning is needed for relevance prediction. |
| Approach: | They propose a set of reasoning-based dense retrieval models trained with data derived from mathematical problem solving using large language models. |
| Outcome: | The proposed model outperforms baselines on the BRIGHT and RAR-b benchmarks and achieves comparable or superior performance while using only 2.5% of the training data used by the concurrent work ReasonIR. |
CoIR: A Comprehensive Benchmark for Code Information Retrieval Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods and benchmarks for information retrieval are inadequately representing the diversity of code in various domains and tasks. |
| Approach: | They propose a benchmark specifically designed to assess code retrieval capabilities. |
| Outcome: | The proposed benchmark aims to invigorate research in the code retrieval domain . it shares the same data schema as other popular benchmarks like MTEB and BEIR . |
Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Modern document retrieval embedding methods typically encode passages (chunks) from documents independently, often overlooking contextual information from the rest of the document. |
| Approach: | They propose a benchmark to evaluate retrieval models' ability to leverage document-wide context. |
| Outcome: | The proposed method significantly improves retrieval quality on ConTEB without sacrificing base model performance. |
Qsnail: A Questionnaire Dataset for Sequential Question Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Questionnaires are a professional research methodology used for qualitative and quantitative analysis of human opinions, preferences, and behaviors. |
| Approach: | They propose a questionnaire-based dataset that consists of 13,168 human-written questionnaires. |
| Outcome: | The proposed dataset contains 13,168 human-written questionnaires gathered from online platforms. |
Query-driven Relevant Paragraph Extraction from Legal Judgments (2024.lrec-main)
Copied to clipboard
| Challenge: | Legal professionals struggle with navigating lengthy legal judgements to pinpoint information that directly addresses their queries. |
| Approach: | They construct a specialized dataset to extract relevant paragraphs from legal judgements based on query . they assess the performance of current retrieval models in a zero-shot way . |
| Outcome: | The proposed model outperforms the current retrieval models in a zero-shot way and fine-tunes them using various models. |
An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Relevance emphasizes the aboutness of a result to a query, while utility refers to the result’s usefulness or value to an information seeker. |
| Approach: | They propose an Iterative utiliTy judgmEnt fraMework to promote each step in Retrieval-Augmented Generation (RAG) they propose to use relevance ranking, utility judgments, and answer generation to prioritize high-utility results over low-utilitity results. |
| Outcome: | The proposed framework improves relevance, ranking, and answer generation on retrieval (TREC DL, WebAP), utility judgment task (GTI-NQ), and factoid question-answering (NQ) datasets. |
X-CoT: Explainable Text-to-Video Retrieval via LLM-based Chain-of-Thought Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-video retrieval systems use embedding models for feature extraction and compute cosine similarities for ranking. |
| Approach: | They propose an explainable retrieval framework upon LLM CoT reasoning to replace embedding models for feature extraction and ranking. |
| Outcome: | The proposed retrieval framework improves retrieval performance and produces detailed rationales. |
NitiBench: Benchmarking LLM Frameworks on Thai Legal Question Answering Capabilities (2025.emnlp-main)
Copied to clipboard
Pawitsapak Akarajaradwong, Pirat Pothavorn, Chompakorn Chaksangchaichot, Panuthep Tasawong, Thitiwat Nopparatbundit, Keerakiat Pratai, Sarana Nutanong
| Challenge: | Large language models (LLMs) show promise in legal question answering (QA), yet Thai legal QA systems face challenges due to limited data and complex legal structures. |
| Approach: | They propose a benchmark which uses Thai financial laws and tax rulings to evaluate Thai legal QA systems. |
| Outcome: | The proposed benchmark compared retrieval-augmented generation and long-context LLM approaches across three key dimensions and found that they improve over naive methods. |